feat: port du compositeur macOS/Linux, capture native, agent d'édition (rebase 1.8.0) - #219
Conversation
|
Important Review skippedToo many files! This PR contains 842 files, which is 692 over the limit of 150. To get a review, narrow the scope: Upgrade to a paid plan to raise the limit. Usage-priced reviews support at most 300 files. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (47)
📒 Files selected for processing (842)
You can disable this status message by setting the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Everything worth keeping from the merge roadmaps, the collision analysis, the two source-repo inventories, the STT migration and POC plans, the recorder roadmaps, the poc-d3d stage logs and the 26 spec files has been folded into the architecture, engineering and testing docs. The rest was history, and git already has it.
ci.yml was scoped to main only, so every PR landing on the long-lived feat/ai-edition integration branch merged with no lint, no typecheck, no tests and no PR-title check. Adds a Docs job running check-docs.mjs. It skips the shared setup action on purpose: the script imports only node builtins, so npm ci would cost a minute for nothing. Drops this branch's comment edit to tests/e2e/roadmap-coverage.spec.ts — #166 deletes that file, and modify-vs-delete is a conflict for no gain.
…ear lint Turning CI on for feat/ai-edition surfaced failures that had been invisible because ci.yml only ran on main. None were caused by the docs work. compositorViewService: the absolute-pin test hardcoded "C:/vendor/ffmpeg", which path.isAbsolute rejects on POSIX, so the pin got nested under the crate dir. Spell it through path.resolve and assert the behaviour instead of one platform's drive letter. The production code was right. whisperServer: two tests wrote a fake helper binary with the default mode, which is not executable on POSIX, so the manager refused it before reaching the assertion under test. Write it 0o755. Lint: biome format across 11 files, plus five dead variables it would only remove under --unsafe. Each checked for side effects first — run_wcpp's reported_rtf was written and never read (rtf is recomputed from wall_s), and e2e-pipeline-smoke keeps its dynamic import, only dropping the unused binding.
… click FloatingInspector floats its "Collapse inspector" button over the reused pane header (position: absolute, right: 12, 26px wide). That lands squarely on whatever the header puts at its right edge and eats its clicks. The Help button was the casualty: rendered on every facet, visible, and unreachable by mouse. CaptionsPane's status icon sits under it too. Fixed where all three headers share a definition rather than nudging the one button: .paneHead now reserves the floated control its own lane. Every pane wearing that class is rendered inside FloatingInspector, so there is no other caller to regress. Caught by the e2e rewrite in the following commit.
The three editor specs were written against the pre-v4 shell and could not pass: they waited on a `timeline-pane` testid that died with TimelinePane.tsx, read data-clip-count / data-zoom-multiplier / data-px-per-sec attributes the v4 timeline never had, and seeded the browser shim through the localStorage keys it stopped reading when it moved to `browser-shim-projects-v2`. Their fixtures were schemaVersion 3, so they exercised the v3→v4→v5 migration path rather than the shape the app actually stores. roadmap-coverage.spec.ts → v4-shell.spec.ts, renamed off the retired roadmap task ids, three behaviours kept and two dropped: - asset file size, now read off MediaStage's cards; the sized and unsized branches share one fixture instead of two page loads - the facet help popover, still live in RightPanes' Pane but reached from the inspector rail rather than a right-panel tab. Its "Settings for X" fallback assertion is gone with the fallback itself — every pane has real help text - ctrl+wheel zoom, which no longer has a px-per-sec ceiling; v4 zooms a nav window bounded at a 2% minimum span, so that is what the test measures Dropped: the ruler hover-guide, which does not exist in v4 (the only ruler overlay left is the pill-drag snap guide), and the chat "applied:" case, which asserted that the browser shim's canned reply carries no tool calls — the mock, not the app, and the positive half is unreachable without a real provider. seed.spec.ts → editor-smoke.spec.ts. Its timeline half was unsalvageable but its preview half still holds, and it seeds nothing, which makes it the only cover for the cold-boot branch (Preview → EditorEmptyState) and for a crash during mount against an empty shim. Now also listens for pageerror, not just console.error — an uncaught mount exception is the failure a boot smoke test exists to catch. diagnostic.spec.ts deleted. It dumped the DOM and a11y tree to find selectors in the old shell and ended in expect(true).toBe(true); the selectors it existed to discover are written down in v4-shell.spec.ts now. Both specs read E2E_BASE_URL, so tests/e2e no longer hardcodes a port. E2E_BASE_URL=http://localhost:5199 npx playwright test tests/e2e/v4-shell.spec.ts tests/e2e/editor-smoke.spec.ts → 4 passed
ExportDialog has rendered MP4 through exportMultiNative since the D3D compositor landed; the mp4 branch returns before ever reaching exportAxcutDocument, so VideoExporter was reachable only from the bench. GIF is the one format the document adapter still renders. Removes VideoExporter, WgslFrameRenderer, audioEncoder, muxer, nativeFrameSink, planarChunkQueue, perfTimings, videoDecoder, asyncVideoFrameQueue, the RenderPlan/audioConcat layer that only fed VideoExporter, and src/bench (every arm drove the web path). The renderer->ffmpeg IPC encoder goes with them: its own header says REFUTED and its only consumer was gated on a localStorage flag nothing sets. resolveCropAt + CropScheduleEntry move to exporter/cropSchedule.ts, which GIF still needs. Also drops ~29 modules with zero importers across components/ui, video-editor and ai-edition. tsc clean; 94 files / 1108 tests pass (baseline 109/1338 - the delta is test files covering the deleted code).
poc/ (WebGPU) and poc-native/ (Rust+wgpu) were both replaced by poc-d3d, which is the compositor that actually ships. Neither is referenced by any build script; poc/ was reachable only from .claude/launch.json. Also removes poc-d3d/spikes (throwaway C++ probes with .bat builders hardwired to a local Visual Studio path), the S0-S8 spike reports and HANDOFF.md for a decision already shipped, the stt-eval bake-off harness whose verdict shipped as electron/native/whisper-stt (REPORT.md kept), an unreferenced 428 KB sample mp4 and a duplicate rec-button.png.
Removes specs/computer-use (20 files scripting RightPanelStack, Bottombar, TimelinePane and CropModal — none of which have a source file), seven root specs each headed "Roadmap status: done", the two STT plans whose own successor declares itself the replacement, an axcut-UX spec for a surface that does not exist post-merge, and a divergence audit whose conclusion is consumed. Dangling links in the surviving docs are fixed. Also untracks .worktrees/wt-9ce78f24/HANDOFF.md: .worktrees/ is in .gitignore, but this file predates the rule and pins a dead branch plus a G:\ path from another machine. Kept deliberately: collision-analysis, openscreen-inventory and axcut-inventory, which ai-edition-roadmap.md pins as deep reference, and design/openscreen-editor.html, which it calls the canonical UI target. design/_ds_bundle.js (checked-in bundle) and _adherence.oxlintrc.json (config for a linter absent from the repo) do go.
gsap, @pixi/filter-drop-shadow, emoji-picker-react and fix-webm-duration
have zero imports across src, electron, scripts and tests; the last is a
stale twin of @fix-webm-duration/fix, which has the real call sites.
@langchain/langgraph, @langchain/langgraph-checkpoint and langsmith have
no direct imports either — all three arrive transitively through
deepagents/langchain/@langchain/core, and the declared ranges sit below
what deepagents requires, so they pin nothing. website/ drops clsx.
The four "Cache caption assets" blocks in build.yml key on
hashFiles('scripts/fetch-caption-model.mjs'); neither that script nor
before-pack.cjs exists, so the key degenerates to a constant and the path
caches nothing. bench:export goes with src/bench.
fixture_dir() defaulted to an absolute path inside a personal git worktree (C:\Users\camil\...\worktrees\prerelease-version-tag-ee96ae). Two paths reached it: create_view fell back to it when no screen_path was given, and the exportNative fallback rendered the fixture whenever the timeline was empty — reachable from ExportDialog on any release build, where that directory does not exist. create_view now requires screen_path and errors without it. The fixture-export entry point goes entirely: ExportTask, the napi `export` binding, the compositor "export" IPC action and contract variant, the service method and the exportNative client. An empty timeline now raises exportDialog.nothingToExport (added to all 13 locales) instead of silently exporting someone else's fixture. exportMulti — the real timeline path — is untouched. cargo check clean; tsc clean; 94 files / 1108 tests pass.
The docs reorg on release/1.8.0 landed after this branch forked, so it documented modules this branch deletes. `docs:check` caught two dead links; reading around them showed the prose was describing the removed renderer-side pipeline, not the native one that ships. - export-pipeline.md: the `RenderPlan` section documented a type that only ever fed `VideoExporter` — on release/1.8.0 `renderPlan.ts` was imported by `documentExporter`, its own test, and `videoExporter` alone, while `ExportDialog` already called `exportMultiNative(clips, path, sceneJson)` direct. The diagram claimed it fed the addon; it never did. Points at `buildSceneDescription` instead, which preview.md already documents. - Drops the per-segment cursor section: `cursorSamplesForAsset` and `segment.cursorSamples` went with the web pipeline, and the addon's export `ClipInput` carries no cursor telemetry (only `createView` takes a cursorPath), so the section described capability the native path never had. - Output sizing said "largest clip" via `pickReferenceDimensions`; `tierOutputDims` feeds the crop-aware SMALLEST clip to `calculateMp4ExportSettings`. Codec crosses as a plain `ExportVideoCodec` string now, not a WebCodecs encoder string. - preview.md: removes the "Path B" section for `PreviewCompositor.tsx`, which that same section already called dead code today.
clamp(value, min, max) was defined privately in 16 files across the renderer, the electron main process and the native glue — byte-identical every time. src/utils/math.ts now owns it plus clamp01; electron already imports from src/, so one home serves both sides. Left alone deliberately: the domain-specific clamps (clampPlaybackSpeed, clampVirtualTime, clampFocusToStage, clampDeltaMs, clampSpanAgainstNeighbours) are real functions, not copies. useTimeline.ts redeclared the same three NaN-guards inside nine separate actions. They are floors, not clamps, so they are hoisted once as finiteSec/finiteMs/finiteFraction rather than folded into clamp. Also drops easeOutExpo, smoothStep and easeInOutCubic from mathUtils — exported, no caller. tsc clean; 94 files / 1108 tests pass.
uuid had five call sites across two files, all v4. crypto.randomUUID() is already used ten times elsewhere in the same tree — renderer runs in a secure context, main runs Node 22. Drops the dependency. findVcVarsAll() was duplicated byte-for-byte (modulo comments) between build-windows-wgc-helper.mjs and build-windows-compositor-addon.mjs, as was the promisified spawn. Both move to scripts/msvcEnv.mjs. The two runInVsEnv bodies genuinely differ (SDK lib compat shims vs cargo) and stay put. The two scripts drop from 197/187 lines to 99/93. Kept deliberately: `motion`. Its spring() is a real integrator (keyframes, velocity, stiffness, damping, restDelta, generator protocol) driving cursor smoothing; hand-rolling one to shed a dependency is writing code, not deleting it. tsc clean; 94 files / 1108 tests pass.
m:ss.t was hand-rolled in Modals (formatTc), V4Timeline (fmt) and operations (formatSec); the auto-hours variant existed twice, byte- identical, as MediaStage's formatTimecode and virtual-preview's formatSeconds. All five now come from timeline/format.ts as formatSec and formatSeconds, with formatMs delegating to formatSec. Left alone deliberately, because they are different formats rather than copies: ExportDialog's formatHms (hh:mm:ss, padded hours, no tenths) and timeUtils' formatTimePadded (mm:ss). formatSize was duplicated between MediaStage and LeftPanel; it moves to utils/formatBytes.ts. NOT converted to Intl.NumberFormat as the review suggested — `unit: "byte"` formats in decimal units and localises the unit name, so a French locale renders 1 GiB as "1,1 Mdo": wrong magnitude, unreadable label. The binary ladder stays. Adds format.test.ts covering the cases the six copies disagreed on (negatives, NaN, Infinity, the hour boundary). tsc clean; 95 files / 1113 tests pass.
sceneDescription hand-rolled a linear-gradient parse (strip parens, flat
comma split, first-token per stop) next to a comment admitting "presets
never contain nested parens". exporter/gradientParser's parseCssGradient
already does this and is strictly better: it tokenises rgba()/hsl() stops
that the flat split would have shredded, and resolveLinearGradientAngle
handles the "to bottom right" keyword directions the regex ignored. Same
180-degree default. All 73 sceneDescription tests pass unchanged.
editorSettings' nextLegacy listed 16 `if (patch.x !== undefined)` lines.
Note a plain {...base, ...patch} is NOT equivalent — it writes undefined
over values the caller left alone — so the keys are filtered first. Added
a test for exactly that; it fails against the naive spread.
tsc clean; 95 files / 1114 tests pass.
…dients
Three of the copies folded into the shared helpers were not byte-identical,
and the differences were the interesting part.
`clamp` — blurEffects and video-editor/types both guarded non-finite input;
the shared version did not, so `clamp(NaN, …)` returned NaN instead of a
floor. That reached the exporter: clampFocusToDepth feeds clampFocusToScale,
which frameRenderer and wgsl/evaluate use to build the zoom transform. The
guard now lives in the shared `clamp` (non-finite -> min), which is
blurEffects' old contract exactly and hardens the other 14 call sites for
free. Focus wants a midpoint rather than a floor — an unknown focus means
"centre" — so it keeps a two-line local, same convention as useTimeline's
`finiteFraction`.
`parseWallpaper` — parseCssGradient is anchored on a trailing ")", so it
rejects strings the `startsWith("linear-gradient(")` branch accepts; a
trailing space was enough. Returning null fell through to the image arm and
handed the native scene a CSS string as a file path. The branch is total
again: no parse means angle 180 and no stops, and the Rust side already
falls back to the layout background on empty stops.
Tests: the NaN cases at the shared definition and at both callers, plus the
rgba()/keyword-direction wins the swap was made for — untested until now —
and the reject path.
- `nextLegacy`'s cast was `Partial<LegacyShape>`, which accepts any key. Pick it down to the patch's own keys so an EditorSettingsPatch field with no legacy counterpart fails to compile, as the 16 `if`s used to (verified: a bogus key now errors TS2344). - build-windows-compositor-addon's `run()` picked up ROOT as its default cwd in the msvcEnv extraction; it was poc-d3d/, which is where cargo has to run for .cargo/config.toml's FFMPEG_DIR to apply. Only call site passes cwd explicitly so nothing broke, but the next one would have. - chat-service now imports randomUUID from node:crypto like codex-session next to it, rather than reaching for the global in main-process code. - Dead JSDoc for the deleted firstTokenOf, stacked on parseWallpaper's own and still claiming the presets never nest parens. Plus two import placements.
document.agent, .preview, .export and .history were declared, defaulted, seeded in every test fixture and written on every edit — and read by nothing. preview.revision in particular was bumped at eight sites and consumed only by tests asserting the bump happened. Removes those fields, the ten schemas behind them (pendingQuestion, preview, exportState, suggestion, agentState, operation, revision, plus the three input schemas with no callers) and their inferred types. timelineOperationSchema STAYS: the review called it dead, but AxcutTimelineOperation has 19 call sites. operationSchema only extended it with suggestion ops nothing dispatches, so that one does go. Old saved projects are unaffected: documentSchema is not strict, so zod strips the now-unknown keys on parse. schema/index.ts 709 -> 604 lines. tsc clean; 95 files / 1114 tests pass.
llm-call.ts opened with "no LangChain dependency" and hand-rolled a multi-provider fetch/SSE client — OpenAI-compat, Anthropic, Codex and Copilot dialects, 736 lines — while the live agent loop went through LangChain the whole time (chat-service.ts -> deep-agent/service.ts). Its only two remaining callers were one-shot prompt->text calls: chat compaction and caption translation. Both now use createOpenScreenChatModel(...).invoke(), which already knew every provider. Removes llm-call.ts, codex-session.ts (a bespoke chatgpt.com Responses dialect reachable only from it) and agent-provider-capabilities.ts, whose per-provider reasoning-effort matrix was duplicated inside deep-agent/. The Copilot runtime-token swap is preserved, and this fixes it rather than porting it: chat-model.ts previously used the PAT directly with a comment saying the refresh "can land as a follow-up", so the MAIN agent path was the broken one. It now exchanges the PAT for the short-lived runtime token, honours the base URL that exchange returns, and sends the editor-identifying headers the endpoint requires. Every caller benefits, not just the two being rerouted. extractDelta (MessageContent -> text) is exported from deep-agent/service rather than re-implemented. tsc clean; 93 files / 1082 tests pass (the delta is the two deleted test files, which covered the deleted code).
addClipBefore, addClipAfter, editClip, splitAndInsert, addTrimAt, updateTrimRange and clearClipSelection were defined, memoised and returned from useTimeline, and called by nothing. insertClipAt, setTrimEntries and updateClipSourceRange already cover every UI path. editClip had its own test suite, which is why it looked alive. The live Edit Clip modal does not use it: NewEditorShell.tsx:1316 calls tl.updateClipSourceRange + tl.updateClipCrop. The tests went with the action. Also removes RightPanes' TimelinePaneBody and the RightPaneId type, both exported and unreferenced — FloatingInspector's Facet union replaced the latter. useTimeline.ts 1192 -> 970 lines. tsc clean; 93 files / 1079 tests pass.
Review follow-ups on the llm-call.ts removal. The reroute to createOpenScreenChatModel was right, but three things did not survive it. `accountId` was threaded from both new call sites into OpenScreenChatModelConfig and then read by nobody. The deleted llm-call.ts set `chatgpt-account-id` from it and called the header required by chatgpt.com/backend-api, so caption translation and chat compaction were newly broken for Codex accounts — the same bug class the Copilot fix addressed, pointed the other way. tsc cannot see it: an optional field that nothing reads still typechecks. caption-translate built a model per batch. For Copilot that is a PAT -> runtime-token exchange against api.github.com on every iteration; at batch size 40 a 400-segment transcript paid for ten of them. Built once, above the loop, with the construction failure kept in the same result shape. extractDelta is now messageContentToText in chat-model.ts. It was exported from deep-agent/service.ts, whose top level pulls `langchain` and the whole agent-tools graph — a lot to import for ten lines of content flattening. chat-service.ts worked around it with a dynamic import; caption-translate.ts did not. Both now import it statically with nothing dragged along. Also normalizes the provider once at the createOpenScreenChatModel entry point. The deleted agent-provider-capabilities.ts resolved `claude` -> anthropic and `gemini` -> google and had tests for it; the surviving deep-agent copy compares raw strings, so those aliases stopped resolving and normalizeProviderId was left with no callers at all. And the surviving copy was not the equivalent duplicate the deletion assumed: isOpenRouterReasoningModel strips the `anthropic/` slug prefix but not `openai/`, and both vendor matchers are anchored at the start — so openrouter + openai/gpt-5 silently got no reasoning wiring. The deleted file stripped both and its deleted test asserted it. Adds the tests back for the module that survived, plus runtime checks for the account header and the alias routing.
migrate.ts still built `agent: {pendingQuestions, suggestions,
lastAppliedOperations}` on every migrated project. zod strips it, so it
was harmless — and it was in shipped code, not a fixture.
transcribe.test.ts still carried agent/preview/export, including a
`preview.playbackRate` that was never in the schema.
These survived because tsconfig.json excludes **/*.test.ts(x) and its
`include` is ["src", "electron"] — no test file and nothing under tests/
is typechecked, and vitest transpiles without checking. So the gate the
deletion leaned on could not see the fixtures it rewrote by hand.
Also drops the comments still describing `preview.revision`, a field that
no longer exists.
(The roadmap-coverage.spec.ts fixture this originally also fixed is gone:
release/1.8.0 deleted that spec in the v4-shell e2e retarget.)
The boolean was declared, defaulted, persisted, migrated and
history-tracked across five files and read by nothing. Its only UI was
RightPanes' TimelinePaneBody, which was already unreferenced before PR 3
deleted it — so the toggle had no way to be set and no effect if it was.
Removes it from EditorSettingsSnapshot, DEFAULT_EDITOR_SETTINGS,
LegacyShape, getEditorSettings, EditorSettingsPatch,
DEFAULT_EDITOR_APPEARANCE_SETTINGS, ProjectEditorState, the
projectPersistence parse branch, EditorState + INITIAL_EDITOR_STATE, and
the v2 migration's editor envelope.
The i18n keys go with it: settings.timeline.{title,waveform,help} in all
13 locales, not the 10 the note listed — vi, zh-CN and zh-TW were
missed. Those three keys were the whole `settings.timeline` namespace and
TimelinePaneBody was its only consumer; the separate top-level `timeline`
namespace (FloatingInspector, V4Timeline) is untouched.
Old projects keep working: migrate spreads input.editor wholesale into
legacyEditor, and legacyEditorSchema is z.object({}).passthrough(), so a
stored showTrimWaveform rides along inertly instead of failing the parse
— and getEditorSettings reads legacyEditor through a loose shape check
that ignores unknown keys. Pinned with a test rather than assumed, since
tsconfig excludes test files and would not have caught a regression here.
tsc clean; biome clean; 95 files / 1101 tests pass.
… wallpapers load
The native compositor is a Rust addon that opens asset paths with `image::open`.
`resolveSceneAssetPaths` handed it paths rooted at `VITE_PUBLIC`, which is
`<root>/public` in dev but `<resources>/app.asar/dist` in a packaged build: vite
copies `public/` into `dist`, so the files are in there, but inside the archive.
Only Electron's patched `fs` reads through an asar, so every bundled wallpaper
and every themed cursor sprite failed to load in an installed app — silently, on
the native side's colour/default-art fallbacks. Picking a cursor theme looked
like it did nothing, and a bundled image background came out a flat colour.
A CUSTOM uploaded background kept working the whole time, which is what pinned
it down: it travels as a `data:` URL the addon decodes in memory, never a path.
`extraResources` already copies both trees to `<resources>/{wallpapers,cursors}`
as real files — electron-builder.json5's own "Asset layout contract" comment
names exactly this, and `ASSET_BASE_DIR` in electron/windows.ts already resolves
them that way for the renderer. So probe the bases in order and take the one
that actually holds the file, the same candidate-list idiom
`ffmpegSharedBinCandidates` uses right below. Existence-probing rather than
branching on `app.isPackaged` keeps `--dir` staging builds working too.
This resolver also feeds `exportMulti`, so the same assets were missing from
rendered exports in installed builds, not just from the preview.
…stops reading as truncated The tilt path (mode 8) decides coverage with a binary `r.z < 0.5` reject, and the 1.5px SDF feather that turns that staircase into an edge sat behind `if (radius_px > 0.0)`. With the Roundness slider at 0 — its default — the plane kept hard aliased edges, which is precisely the failure this branch's own comment describes: "des arêtes de couteau qui coupent le contenu en pleine phrase, et ça se lit comme une troncature plutôt que comme une inclinaison". Hence the report that the 3D zoom looks cut off, but only below some minimum corner radius. `sd_round_rect` degenerates to a plain box SDF at r = 0, so the guard bought nothing: drop it and clamp with `max(radius_px, 0.0)`. The mode 12 shadow already applies its radius unguarded for the same reason. The geometry was never the problem — `every_preset_stays_inside_the_original_rect` already holds, so nothing was being clipped by the frame.
…en a file is refused
`handleFileSelected` gated custom wallpaper uploads on
`file.type.startsWith("image/")`, so a file the browser reports no MIME type for
— which Windows does for some files and some locales — was dropped on the floor,
with no toast, no message, nothing. The user picked a PNG and the pane simply did
not react.
That exact case had been fixed once, in "Allow PNG custom background uploads":
`isSupportedBackgroundImageType` fell back to the file extension when `type` was
blank, and its test named a real offender (`生成画像1.png`). The module was never
imported by the upload path though — only by its own test — so the 2026-07-26
reorg removed it as dead code, correctly, and the repo lost the fallback while
keeping the weaker check that needed it.
So restore the fallback where the upload actually happens, and derive the
`accept` filter from the same two lists the validation uses, since those had
already drifted apart once (the accept string was an inline copy of the deleted
module's constant). An explicit non-image type is still refused — only a blank
one earns the extension fallback, so `notes.txt` renamed to `notes.png` does not
get through.
Rejections and unreadable files now raise a toast, localized across all 13
locales. Tests are the recovered cases plus a component test asserting the toast
actually reaches the user, which is the half that was really missing.
layer.wgsl has had the mode-2 drop shadow since the port began — a rounded-rect SDF whose alpha falls off across the spread. What was missing was the Rust half: compose_frame never built a mode-2 LayerCB and never issued the two draws, so `cfg.shadow` and `lp.shadow_scale` were read by nobody and the Shadow slider did nothing at all on Linux. Both shadows use the tuning fractions already shared with macOS/Windows (SCREEN_/WEBCAM_SHADOW_SPREAD_FRAC and friends in frame_geometry), so they are the same relative size on all three platforms at any output resolution. Each is drawn immediately before the layer it belongs to: under that layer, over what is behind it. The camera shadow is skipped in the dual-frame and vertical-stack presets — the camera is tiled flush against the screen there, and a shadow between them would draw a seam. Same condition as macOS. Adds a rendered-frame test that exercises this together with the two previous fixes: shadow on, camera present, and a text annotation with a background. It asserts the annotation's plate is actually on screen — counting pink pixels in the region — which fails outright when the plate is not drawn, rather than only being visible to someone opening the PPM.
Ports the perspective-rotated screen layer, its projected-quad shadow,
and the tilted cursor. Verified independently of the agent that wrote it:
99 lib tests and 8 rendered-frame tests green in this checkout, and the
renders inspected by eye — the iso preset is an unmistakable trapezoid
with the right edge receding, rounded corners following the tilt, and the
full source content present (so crop and zoom really are applied to (s,t)
rather than sampled raw).
None of the maths is new: regions.rs was already cross-platform and its
tilt tests already ran on this host. What was missing was GPU code and
uniform packing.
Two porting traps avoided, both called out in the Metal source:
- mode 8 has NO dst_prev clip test, because dst_prev.xy holds plane_px
in pixels there rather than a 0..1 rect;
- mode 12 reads its spread from mb.y, not fx.x as mode 2 does.
The MSL's two runtime-indexed local-array loops are hand-unrolled rather
than translated literally: naga 24 emits invalid SPIR-V for that pattern
and RADV segfaults inside vkCreateGraphicsPipelines. The HLSL uses a
dynamic array index for sd_convex_quad, which is exactly why the Metal
was the right source to translate from.
The new branches sit BEFORE the mode-2 catch-all `else`, which is
unbounded — anything falling through it silently renders as a drop
shadow.
Mode 13 was optional and got done anyway, because modes 8 and 12 make
tilted zooms reachable and the old cursor path made the pointer vanish
outright under one. That would have been a regression introduced by this
very commit.
Assertions are geometric rather than existence checks: the upright
reference's top edge is flat to 0px while iso/left/right all swing 27px,
and left/right land 1px apart, which is the mirror symmetry the presets
should produce.
Known gap: `regions.rs:956` calls the left/right presets "pure Y
rotations", but they are [-8,-16,-1] and [-8,16,1] — all three axes
nonzero. No preset is truly affine, so quad_inverse_bilinear's
degenerate `abs(k2) < 1e-5 * abs(k1)` branch is ported faithfully but
unexercised by these tests.
… that ignored their own scene I told the user the circle mask was missing on Linux. It is not — I was wrong, and the way I was wrong is worth recording. Two rendered-frame tests I wrote called set_scene() without set_live_params(). compose_frame reads LiveParams, not the raw scene, so webcamShape/padding/effects never arrived: the scene parsed fine and was then ignored, and the renders showed the DEFAULT "rounded" shape. I read those rounded corners in a downscaled frame as "rectangular" and reported a missing feature. This gotcha is documented — it is the first item under "Test gotchas" for this crate — and I walked into it anyway. The new test measures instead of eyeballing. It renders the same scene three times (no camera / rectangle / circle) and diffs each against the no-camera pass, so the diff IS the camera's footprint including its mask, with no need to guess where plan_frame put it or to tell camera pixels from background. Then it checks the bounding box's fill ratio, which separates the two shapes unambiguously: rectangle 256x192 AR 1.333 fill 0.993 circle 256x256 AR 1.000 fill 0.798 (pi/4 = 0.785) So the circle both forces a square box and crops to a real disc. Had this test existed with set_live_params missing, it would have reported identical numbers for both shapes — which is exactly what it did before I fixed it, and is what made the omission obvious.
The two native references genuinely disagree here: Windows centres with DWRITE_PARAGRAPH_ALIGNMENT_CENTER (text_windows.rs:175-176) to match the web overlay's `alignItems: center`, macOS does not, and the Linux port had copied macOS — so text sat at the top of its plate. The web is the product intent, which the user confirmed. The block height is measured from the last laid-out run rather than counting runs times line_height: cosmic-text splits one logical line into several runs when it wraps, so counting runs would overestimate the height of any text that overflows its width and push it up out of centre. Text taller than its box stays anchored at the top instead of overflowing upward, where it would be cropped away entirely.
…hine GGML_NATIVE was ON under a comment claiming it produced a "+avx2 +fma +f16c" baseline. It does not: ggml turns GGML_NATIVE=ON into `-march=native`, so the CPU backend targets whatever CPU ran the build. build-whisper-stt.yml builds the shipped binaries on GitHub-hosted runners, whose x64 images are Xeons with AVX-512 — so a published artifact could carry AVX-512 and die with SIGILL on every AMD before Zen 4 and every Intel consumer part since Alder Lake, and the baseline shifted silently with whatever hardware picked up the job. Same class of bug as the OpenSSL note: an artifact whose contents depend on the machine that built it. Turning GGML_NATIVE off is necessary but not sufficient. ggml derives its instruction-set options from INS_ENB via `option()`, which only seeds a value the first time a build tree is configured — a cache written while NATIVE was ON keeps GGML_AVX2=OFF, and the result is a plain x86-64 binary with no AVX2 at all. Verified: that path produced zero %ymm instructions. So pin the baseline explicitly instead of inheriting it. Ships SSE4.2 + AVX + AVX2 + BMI2 + FMA + F16C, x86-64 hosts only, AVX-512 deliberately excluded. That is the Haswell/Zen 1 floor the installer already requires. OSC_NATIVE_CPU=ON restores -march=native for local benchmarking. Verified on linux-x64: `-msse4.2 -mf16c -mfma -mbmi2 -mavx -mavx2`, 12744 %ymm instructions, 0 %zmm, and the helper still transcribes on Vulkan.
build-whisper-stt.sh stages a symlink farm on Linux and macOS (libggml-vulkan.so -> .so.0 -> .so.0.15.1), and plain `cp` follows every link named on the command line, so each one became a full copy of its payload. libggml-vulkan.so.0.15.1 is 60 MB on its own: measured, the bag was 188 MB instead of 63 MB and the uploaded tarball 59 MB instead of 20 MB. gzip cannot recover any of it — its 32 KiB window is far too small to dedupe copies that sit 60 MB apart. `cp -a` preserves the links; tar keeps them from there (no -h) and untars them intact on the consuming side. Verified end to end on linux-x64 with the real libraries: cp -a -> tar czf -> tar xzf leaves 12 symlinks and the extracted helper still answers GET / and transcribes. `dir/.` rather than `dir/*` so the copy does not depend on the shell's glob skipping dotfiles. -v is kept: the step's log output was the point of it. This shrinks the artifact and the CI download only. The installers are unchanged, because scripts/stage-whisper-stt.sh:65 is a second plain `cp` that re-expands the farm before electron-builder runs. Making that one -a as well would cut ~131 MB from every package — electron-builder recreates symlinks (builder-util/out/fs.js:263) and squashfs stores them natively (verified: a 61 MB farm becomes a 19 MB image), but deb and pacman go through fpm, whose behaviour here I could not verify, so that step wants a real package inspection rather than an assumption.
The user reported the custom cursor only appearing once they opened the cursor settings panel. The cause is broader than the cursor and worth stating plainly: NOTHING pushed the complete param set. Each group was pushed only by the mount effect of the panel that owns it — VideoEffectsPane, LayoutPane, CursorPane — and the inspector renders exactly one panel at a time, defaulting to "effects". So on load the layout and cursor panels had never mounted and their params sat at the addon's compiled-in defaults: cursor size 1.0 for a project storing 3.0, smoothing 0 for 0.67, motion blur 0 for 0.35, webcam shape and mirror likewise. Opening the panel pushed them and it looked like the panel had switched something on. The store's lastParams replay could not save this: it replays keys that were pushed at least once, and these were never pushed at all. pushAllNativeParams() now sends all fourteen keys the addon understands (live.rs set_param_bool/num/str), called from NativeCompositorOverlay — the one component that exists regardless of which panel is showing, and that already owns the view and the scene push. No viewId guard: setNativeParam memoises and setCurrentNativeViewId replays, so mount order does not matter. It cannot fight a slider drag either, because setLive goes through setDocument, so the overlay's document selector already carries the new value on the same tick. The three mount-sync effects are removed rather than left as belt and braces — duplicated pushes of the same value are how the two paths would silently drift apart. UNITS. The helper sends RAW values, matching the old mount effects and NOT the sliders: SliderCell displays cursor.size * 10 and divides by 10 in its handler, with * 100 for smoothing and motion blur. Sending slider-space values here would scale the preview by 10 or 100 on load. The two base-unit constants moved to src/native/paramUnits.ts so the helper and the panels cannot disagree about them. Also fixes the Camera Shape row overflowing its panel. `repeat(4, 1fr)` is `minmax(auto, 1fr)`, so each track's minimum resolves to its content minimum, and "Rounded" is an unbreakable word: four buttons at 64.83px plus three 8px gaps need 283.3px where the panel has 234, so the tracks refused to shrink and the last button was clipped. `minmax(0, 1fr)` plus minWidth:0 on the buttons lets them shrink. The new test asserts NO addon key is missing, transcribing the list from live.rs — a test that checked only the keys we happen to push would have passed against the bug.
Two independent bugs made a Linux recording sound empty. Both are fixed here because either alone still leaves silence. WRONG MICROPHONE. The renderer sent no device at all, so the helper passed no PW_KEY_TARGET_OBJECT and PipeWire linked the stream to the session DEFAULT source. On the machine this was found on that is the headphone jack with nothing plugged into it; the user's built-in microphone was a different node. Measured: the recorded track sat at -61 dB mean while the chosen microphone gave -20 dB. The HUD's level meter moved the whole time because that is getUserMedia, which does honour the picker. This was a documented gap, not an oversight — nativeLinuxRecording.ts said the browser deviceId and a PipeWire node.name are unrelated namespaces and left the field empty. Documenting a hole is not filling it, and I shipped it saying audio worked. There is no id to look up, so the helper now enumerates the graph itself (pw_registry, new in pw_audio.c) and matches the label Chromium reports against node.description — which on a PipeWire system is the same string. Exact node name, then exact description, then containment both ways with the longest description winning so the result cannot depend on registry announcement order. No match means no target, i.e. the old default-source behaviour, plus a warning saying so. Verified live: label "Family 17h/19h HD Audio Controller Digital Microphone" resolves to alsa_input.pci-0000_03_00.6.HiFi__hw_acp6x__source. ONE TRACK, NOT TWO. The MP4 carried system audio and the microphone as separate AAC tracks, following macOS on the grounds that the export mixes every track it finds (audio.rs). The preview does not: it plays an HTML5 <video>, which plays only the default audio track and cannot be told to switch, because Chromium does not implement the audioTracks API. System audio was written first and flagged default, so with nothing playing the preview was silent while the microphone sat in a track nothing would ever select. The Windows helper has always written one mixed track (mf_encoder.cpp has a single audioStreamIndex_ fed by AudioMixer); this now matches it. macOS still has the two-track bug. Mixing consumes min(available) across inputs so neither runs ahead, which taken literally means an unplugged microphone freezes the whole track — its min stays zero forever. AUDIO_STARVE_SAMPLES (a quarter second) breaks that: past it the mix takes the longest input and lets the silent one contribute zeros. Contributions are summed and clamped ONCE at the end; clamping per input would attenuate the mix whenever one source alone is near full scale.
The line that answers "that is not my microphone" was emitted as a `debug` event, which the emitter drops unless OPENSCREEN_PIPEWIRE_DEBUG is set. The app never sets it. So when the user recorded, the resolved node was absent from their log and the fix had to be inferred from a volume measurement instead of read directly. It is now a first-class `audio-source` event carrying the role, what the app asked for, and the PipeWire node actually used — logged unconditionally by the capture session, with a warning when a requested device matched nothing and the session default was recorded instead. Diagnostics for a failure mode that is invisible to the user (audio from the wrong device sounds exactly like a quiet room) do not belong behind a debug flag.
Adding a text annotation created it empty, and an empty annotation renders nothing — so the user added a region and saw no change on the canvas. The inspector's placeholder is CSS ghost text: it never reaches `content`, so it never reached sceneDescription and never reached the compositor either. `annotation.defaultText` added to all thirteen shipped locales (the parity test walks every folder and fails on any key `en` defines alone). The values are proposals, not verified project wording — no "Hello" existed anywhere in the locales to copy a register from. Worth a native speaker's eye, particularly the CJK ones where "こんにちは" may read as more conversational than a UI default should. A default in a script the machine has no font for would render as nothing — the exact bug this is meant to fix, but silent and only for some users. So a compositor test rasterises all thirteen through the real Linux text path and asserts each produces ink. On this machine all pass, Arabic, Japanese, Korean and Chinese included: 677 to 2254 px each. It documents what the running machine lacks rather than pretending to guarantee font coverage everywhere. useTimeline now reads the locale, so its 17 tests need the I18nProvider. One `renderTimeline` helper covers 16; the render-count test keeps its own renderHook because it counts renders itself.
A recording made with the camera on opened in the editor showing the SCREEN video inside the camera's PiP box. Two independent defects, one behind the other. THE CAMERA WAS NEVER LINKED. `cameraTrack.offsetMs` is an integer in the document schema, and both native capture paths derive it from two `performance.now()` readings — a clock with 100 µs resolution, so the subtraction lands on something like -192.80000000447035. `parseDocument` then threw inside the camera auto-link, the catch treated it as a lookup failure, and the asset was saved with `cameraTrack: null`. Four of the last five recordings on this machine lost their camera that way; the fifth measured -243 exactly and kept it, which is what made the bug look intermittent rather than mechanical. Rounded at the recorder so new manifests are clean, and again in `addAsset` so recordings already on disk recover. THE MISSING CAMERA RENDERED AS THE SCREEN. `open_and_seek_clip` opens the screen file as the webcam decoder when the webcam path won't open, so the "webcam" always has frames; `LiveParams::has_webcam` is what says whether they are a camera. It was computed as "webcam path != screen path", which is true for the empty string the app actually sends for an asset with no camera — and the Linux backend never read the flag at all, unlike Windows and macOS. Either bug alone would have drawn the recording into its own corner. Both fixes are covered by tests that fail without them: the rounding in `projectStore.test.ts` and `webcamOffset.test.ts`, the rendered frame in `compose_linux.rs`, which asserts strict equality with the no-webcam preset rather than a threshold. macOS shares the offset defect line for line and is fixed here too.
The Linux compositor only ever drew ONE of the four annotation kinds. A
project could carry arrows, blur masks and images all the way here — the
scene parsed them, `SceneAnnotation` had the fields — and the render loop
skipped anything that wasn't `text`, silently. Worse for the blur: an
annotation whose whole job is to hide something rendered as nothing at
all, so the export showed in the clear exactly what the user marked as
private.
Ported from `compositor_macos::draw_annotations`, mode for mode:
- mode 9 (figure): three round-capped SDF segments, the same trace as
`ArrowSvgs.tsx`. `regions::arrow_local_geometry` was already shared
and tested, so the shader receives endpoints, not a direction to
guess at.
- mode 10 (blur): reads a mipmapped copy of the composed frame. wgpu
has no `generate_mipmaps`, so the pyramid is built by one fullscreen
pass per level with the existing `fs_copy` pipeline. It is the
pyramid that makes the blur — a few offset taps only produce ghosting.
- mode 7 (image): contain-fit, never stretched, cached by annotation id
rather than by path (an image annotation carries a multi-megabyte
data URI, which would be rehashed every frame as a map key).
Annotations now have their own render pass. They must: the blur snapshot
has to be taken after the screen and camera are down but before the first
annotation, and a pass cannot sample the target it draws into. Taking it
once per frame is also what keeps two overlapping blurs from sampling
each other.
`text_anim.rs` was ported, unit-tested and reachable, but nothing on
Linux called it — animated text simply appeared all at once. It now
drives opacity, scale, translation and the typewriter reveal, with the
background plate following the same opacity so a fade doesn't flash a
solid rectangle first.
The four new tests measure on the GPU rather than assert that a draw was
issued: an arrow must cover less than half its box (an unknown mode falls
through to the shadow branch and fills it), a blur must destroy the high
frequencies under it, a mosaic must produce equal neighbours, a 4:1 image
must still measure 4:1, and a fade at 0 ms must be invisible.
Note for anyone writing more of these: `compose_frame`'s third argument
is a FRAME NUMBER, not seconds. Passing 1.0 puts the timeline at 16 ms,
which silently moves every annotation outside its visibility window — the
first version of the animation test passed for that reason and not
because the fade worked. `set_timeline_time` pins it honestly.
None of these were visible before the rebase: each one is a collision
between this branch and something v1.8.0 changed underneath it.
Windows `cargo check` — v1.8.0 made `pipeline_windows::Decoder`
`pub(crate)`, and `tests/compose_linux.rs` imports `pipeline::Decoder`.
Files under `tests/` compile on every platform regardless of what they
test, so a Linux-only test broke the Windows build. Gated with
`#![cfg(target_os = "linux")]`, mirroring the `#![cfg(windows)]` that
`warp_device_cannot_decode.rs` already carries for the same reason.
Lint — one `path.join` in the pipewire helper's build script was split
across lines where biome wants it on one. Formatter only.
Test typecheck — this job fails when the error count grows past a
baseline. v1.8.0 measures exactly 71, the baseline in ci.yml; this branch
measured 81. The ten new ones had three causes, and none of them was a
real type problem in the code under test:
- `nativeCompositorStore.test.ts` (7): the `setCompositorParam` mock was
`vi.fn(() => …)`, so its inferred call signature took no arguments and
`mock.calls[i][1]` indexed an empty tuple. Spelling out the parameters
types the recorded calls.
- `pipeWireCursorRecordingSession.test.ts` (2): `actual.default` on a
node builtin. The default export exists at runtime — it is the CJS
namespace, and the module under test may import it — but `@types/node`
declares only the named exports, so it is read through an index type
rather than dropped from the mock.
- `projectStore.test.ts` (1): `schemaVersion: 6` in an unannotated object
literal widens to `number`, and the document type pins the literal `6`.
`as const` fixes the fixture, and with it the five sibling errors that
were already in the baseline.
That last one takes the count to 66, so BASELINE drops to 66 — the job's
own comment asks for it, and leaving it at 71 would bank room for five
future regressions to slip in unnoticed.
The `defaultOf` helper this started as is worth a warning: `vi.mock` calls
are hoisted above every top-level statement, so a module-scope helper is
still in its temporal dead zone when the factory runs and the mock dies
with "Cannot access 'defaultOf' before initialization". A `type` alias is
erased and safe; a `const` is not.
1220 renderer tests pass, 112 + 17 + 3 Rust, lint clean, tsc clean.
`createDeepAgent` empile trois middlewares inconditionnels — filesystem, todo,
sous-agents — qui ajoutaient HUIT outils (ls / read_file / write_file /
edit_file / glob / grep / write_todos / task) adossés à un backend mémoire VIDE,
plus ~5 800 caractères de prompt promettant un système de fichiers que cette app
n'a pas. Rien ici ne s'en servait, et le coût n'était pas cosmétique.
Interrogé sur la télémétrie curseur d'un projet, le modèle lançait
`ls {"path":"/"}` puis `glob {"pattern":"**/*"}` contre ce bac à sable vide et
rapportait, de bonne foi, que le projet ne contient aucune donnée de pointeur.
Une preuve d'absence fabriquée : le sidecar .cursor.json existe et le
compositeur en charge les échantillons dans la même session. Mesuré au banc,
le scénario qui pose la question tombait à 0 % de réponses honnêtes sur 3 tours.
`createDeepAgent` n'expose aucune option pour retirer ces middlewares (ils sont
dans `REQUIRED_MIDDLEWARE_NAMES`), donc le correctif est de cesser d'y passer :
`createAgent` est ce qu'il enveloppait, sans le bac à sable. Le modèle reçoit
désormais 19 outils, tous ceux d'OpenScreen, et un prompt de 4 591 caractères.
Le reste du lot corrige ce que la même campagne de mesure a mis au jour :
- La table depth→échelle annoncée au modèle était fausse aux deux bouts. Le
prompt affirmait « depth 1–6 maps to 1.0×–3.5× », une formule (`depth/2+0.5`)
purgée de deux sites de rendu et survivante ici ; la vraie table va de 1,25× à
5,0× et depth 3 rend 1,80×. Le légende est maintenant DÉRIVÉE de la table, et
les résultats d'outils portent `renderedScale`.
- `replaceTimeline` détruisait sur une demande anodine : sollicité pour
réordonner deux clips — ce qu'il ne peut pas faire, puisqu'il trie ses
intervalles — il re-frappait les ids et supprimait un trim, puis le tour
annonçait le trim préservé. Il refuse désormais quand l'opération perdrait un
clip ou un trim, et `moveClip` fait le réordonnancement en préservant ids,
plages source, trims et effets ancrés.
- `allowAgentEdits` n'était appliqué nulle part : décoché, l'agent éditait sans
demander, 0/3. La garde est dans l'exécuteur, doublée d'un bloc de prompt —
le banc note les appels ÉMIS, et un tour qui tente trois écritures refusées
reste un tour qui a tenté.
- `hasCameraTrack` et `hasCursorTelemetry` entrent dans le snapshot. Le modèle
posait une région caméra plein écran sur un projet sans caméra (33 %).
Mesuré à n=20, effet minimal détectable 31 % : consentement 33→100 %,
caméra 33→100 %, depth 60→98 %, honnêteté sur le curseur 0→91 %.
…ct d'un détecteur Le modèle n'avait aucun accès à la télémétrie curseur alors que le sidecar existe et que le compositeur le charge. Un premier jet exposait le DIGEST de `buildZoomSuggestions` — la liste des moments où le pointeur est resté immobile. C'est la mauvaise idée, et la mesure le montre : ce détecteur plafonne le modèle à son propre rappel. Sur un screencast réel de 66 s annoté par son auteur, il retrouve 6 zones d'intérêt sur 6 mais produit 8 faux positifs sur 16 arrêts détectés, et il est aveugle par construction à la seule zone qui compte vraiment ici : celle où l'auteur BALAYE lentement une image en la commentant, le curseur parcourant 30 % de la largeur en 8,9 s à hauteur constante. Aucun réglage de seuil ne l'attrape, puisque le curseur bouge franchement. Un modèle nourri de ce digest ne peut donc pas zoomer là, quelle que soit sa qualité. `getCursorTrack` rend donc l'OBSERVATION : les positions échantillonnées, le temps source et le temps virtuel, et l'index de forme du pointeur. Le sous-échantillonnage est une question de résolution, pas d'interprétation — chaque point retenu est un échantillon réel, rien n'est résumé, et tout changement de forme survit à la réduction parce qu'un changement de forme est un événement observé et non un verdict sur cet événement. Sur cette prise : 1521 échantillons → 356 points à 5 Hz, dont 56 gardés pour un changement de pointeur. Le lecteur de sidecar sort de `ipc/handlers.ts` vers `electron/media/` — Node pur, injectable, testable — parce que l'agent avait besoin de lire le même fichier et ne pouvait pas importer un handler IPC. Trois états distincts sont préservés : présent, absent pour de bon, ou illisible d'ici — la différence entre « ce projet n'en a pas » et « je n'ai pas pu regarder ». Au passage, un crash au démarrage : `handlers.ts` capturait `RECORDINGS_DIR` au chargement du module alors que `main.ts` l'importe en retour. Zone morte temporelle, l'app mourait avant sa première fenêtre — et la suite unitaire n'y voyait rien, 1308 tests verts sur un binaire qui ne démarre pas.
Le bouton demandait deux choses au modèle : placer des zooms sur les moments où le curseur s'attarde, et couper les temps morts. Mesuré sur un enregistrement réel de 66 s dont l'auteur a annoté ce qu'il faisait, il fait la seconde très bien et la première mal. Les coupes, sur deux tours : ONZE trims, tous strictement à l'intérieur d'un vrai silence, avec une marge constante de 6 à 33 centièmes — le choix prudent, celui qui n'ampute aucune attaque de mot. Zéro seconde de parole détruite. Et un refus argumenté de couper la longue plage finale, pourtant immobile à l'écran, parce que la narration y est continue : « cutting it would break the demo ». Les zooms, mêmes tours : 7 focus sur 9 ratent la position réelle du curseur dans leur propre fenêtre, trois d'entre eux de plus d'un tiers d'image. Il les place d'après ce que RACONTE le transcript, pas d'après où ÉTAIT le pointeur — son propre commentaire le trahit, il annonce un zoom sur « Iceman, Views » cinq secondes avant que ces mots soient prononcés. Il touche les six zones annotées, mais en zoomant 38 % de la vidéo : c'est de la couverture, pas de la détection. Demander les deux d'un souffle achetait donc des zooms mal visés au prix de la crédibilité des coupes. Le wizard curseur, juste à côté dans le même menu, fait déjà la passe de zoom depuis la même télémétrie et de façon déterministe. Un wizard programmatique pour les zooms, un wizard IA pour les coupes. Le libellé suit dans les 13 locales : « Smart zooms + cuts » devient « Smart cuts ». Rien n'est retiré au modèle — la trajectoire reste exposée par `getCursorTrack`, et le scénario `real-wizard-enhance` du banc est ce qui montrera quand il saura la lire.
Faire tourner l'agent sans interface, pour itérer sur les prompts et sur le contexte qu'on lui donne. Deux axes notés séparément et jamais moyennés : le COMPORTEMENT (le texte final — refuse-t-il quand il manque une donnée, invente-t-il, décrit-il le projet correctement) et le DSL (les appels réellement émis et le document qui en sort). La porte est `min(comportement, dsl)` : un DSL parfait accompagné d'un mensonge échoue, une réponse polie qui édite ce qu'on lui a interdit échoue aussi. L'axe DSL se note sur le fil et sur le document APRÈS, jamais sur le `resultJson` d'un outil : le flux d'événements UI double chaque appel mutant, ne porte pas d'identifiant et code `ok=true` en dur, et les outils renvoient les bornes DEMANDÉES plutôt que celles obtenues. Six oracles de qualité éditoriale, déterministes, parce qu'une note reproductible vaut mieux qu'un juge : secondes de parole détruites — avec les mots amputés nommés, plus parlant qu'un nombre —, précision de coupe signée qui distingue la marge prudente de l'empiètement, couverture des silences en traitant les bords à part, fragments orphelins, placement des zooms en précision ET rappel séparés, et opérations hors périmètre. Chaque tour est persisté — requêtes, document avant/après, texte final. Ce n'est pas du confort : lors du premier passage sur la prise réelle, trois répétitions sur cinq n'ont rien produit, et sans les tours bruts j'aurais conclu que le modèle refusait d'agir. Le journal disait autre chose — un timeout du banc à 120 s, toujours au même endroit, juste après un résultat d'outil de 24 Ko. Les fixtures d'évaluation ne sont pas versionnées (voir .gitignore) : ce sont des enregistrements réels d'utilisateurs, avec leur voix transcrite. Le checklist e2e gagne au passage un contrôle sur le bouton de transcription, qui ne dit rien quand le binaire Whisper est absent — ni message, ni état d'échec, ni une ligne de log.
…logue v4 `testId-export-button` vivait dans `src/components/video-editor/SettingsPanel.tsx`, supprimé quand le shell v4 est devenu le seul éditeur. Il ne reste nulle part dans `src/`, donc ces deux assertions ne peuvent plus passer : la spec échoue sur la CI Windows depuis, pour une raison qui n'a rien à voir avec la capture native qu'elle est censée couvrir. Même sélecteur par rôle que `gif-export.spec.ts`, qui tourne vert sur cette machine. `exact: true` reste nécessaire — sans lui il capterait aussi les « Export MP4 » / « Export GIF » du dialogue. NON VÉRIFIÉ sur sa plateforme : cette spec se skippe sur darwin (`test.skip(process.platform !== "win32")`), donc seule la CI Windows dira si le reste du parcours tient. Le changement se limite au sélecteur.
…r le couperet du banc Deux corrections, dont une de mon propre diagnostic. LE TIMEOUT. J'avais conclu que le track de 24 Ko faisait échouer le tour trois fois sur cinq. C'était une corrélation — les échecs tombaient juste après l'appel à l'outil — servie comme une cause. Les durées disent autre chose : les deux tours réussis prennent 117,0 s et 112,5 s, les trois échecs tapent exactement 120 000 ms. Le couperet était posé 3 à 7 secondes au-dessus de la durée normale d'un tour ; ce n'était pas le modèle qui renonçait, c'était le banc qui mesurait son impatience et l'imputait au modèle. Porté à 300 s. Le contexte n'était pas en cause de toute façon : le tour entier fait ~26 000 caractères, soit ~6 500 tokens. Ce qui prend deux minutes, ce sont ses 19 appels d'outils EN SÉRIE — six addTrim et neuf addZoom émis un par un, chacun un aller-retour complet. C'est la piste à traiter, elle est documentée à part. LES KEYFRAMES. Le track reste réduit, mais pour ce que ça vaut vraiment : de 24 238 à 7 797 caractères, soit désormais SOUS le transcript (10 496) au lieu de 2,3× au-dessus. Deux gains sans perte d'information. `virtualSec` quitte les points quand il égale `atSec` — 28 % du payload à répéter le timestamp à côté de lui-même. L'enveloppe le dit une fois via `virtualEqualsSource`, et le champ revient par point dès qu'un clip est déplacé, coupé ou réordonné et que les deux axes divergent. Et l'échantillonnage devient une réduction en keyframes au sens de l'animation : on garde les points sur lesquels le mouvement tourne, l'interpolation remet les interframes. 1521 échantillons deviennent 148 points. Rien de ce qu'aucune interpolation ne rejoue n'est perdu : changement de forme du pointeur, événement non-move, et bornes de tout arrêt de plus de 3 s — un curseur immobile doit se lire « toujours là », jamais « plus de données ». Un piège d'implémentation, gardé dans le code parce qu'il est invisible : simplifier la TRAJECTOIRE plutôt que les courbes x(t) et y(t) paraît équivalent et ne l'est pas. Un curseur qui part et revient par le même chemin ne s'écarte pas de la corde, donc l'aller-retour s'efface et l'interpolation jure ensuite qu'il n'a jamais bougé. Mesuré : 0,380 d'image d'erreur de reconstruction pour une tolérance de 0,02, contre 0,084 en simplifiant par axe. La version fautive était la plus compacte (4,4 Ko) et la plus séduisante. Au passage, deux bornes que les tests ont attrapées avant qu'elles ne sortent : le plafond de points était défait par la règle de mouvement (2329 points rendus pour un budget de 400), et quand le plancher d'écart dépasse à lui seul le budget, un `|| wantedHz` écrit comme garde anti-division-par-zéro faisait REMONTER le débit au maximum au lieu de l'abaisser.
…édia Un trim était stocké en temps source PAR ASSET, sans identité de clip. Dès que deux clips tirent du même média (clip dupliqué, même enregistrement posé deux fois), ils partagent le même espace de coordonnées : « sur quel clip est cette coupe ? » n'avait pas de réponse, et chaque lecteur en inventait une. Un seul défaut, trois symptômes : - transcript : la coupe apparaissait sur LES DEUX clips ; - règle : la pastille était dessinée sur le PREMIER ; - lecture/export : la plage était retirée des DEUX clips. AxcutTrimRange gagne `clipId` — `startSec`/`endSec` SONT déjà la fenêtre source, donc c'était la seule pièce manquante pour rejoindre l'ancre de clip v5 que zoom/annotation/speed ont depuis longtemps. `trimAppliesToClip` est désormais la définition unique par laquelle passent buildClipSection, trimToTimelineSpan et resolvePlaybackSegments. Un trim sans `clipId` garde le sens historique par asset : les documents existants se lisent exactement comme avant. Schéma 6 -> 7. La migration VENTILE au lieu de choisir : un trim stocké devient une ligne ancrée par clip couvert, chacune bornée à la fenêtre de son clip. Un trim v6 coupait réellement tous les clips de son asset, donc le rendu ne bouge pas — mais les lignes sont maintenant adressables séparément, ce qui permet de supprimer la copie posée sur le clip qu'on ne visait pas. Un trim ne couvrant aucun clip est conservé tel quel plutôt que supprimé. Corollaires du modèle : `duplicateClip` copie explicitement les trims ancrés sur la copie (avant c'était un effet de bord du match par asset), et `removeClip` supprime ceux du clip effacé pour qu'un jumeau du même asset ne les ressuscite pas. --- Transcript : la même ambiguïté un cran plus haut Un transcript appartient à l'ASSET, donc deux clips sur un média projettent le même AxcutWord deux fois : `word.id` nomme un instant dans le média, pas un élément à l'écran. `ClipWord.id` = `clipWordId(clip.id, word.id)` devient l'identité à l'écran (clé React, data-word-id, surlignage du mot lu, ancre du curseur), et `findCueWordId` choisit sa section par `cue.clipId`. Les jetons [silence] rendent ce cloisonnement inconditionnel : withSilenceGaps les numérote à partir de 1 dans chaque clip, donc `silence_1` existait dans tous les blocs. --- Deux défauts adjacents, trouvés en vérifiant Cliquer un mot du transcript appelait `onSeek(word.startSec)` — du temps SOURCE — alors que handleSeek attend du temps TIMELINE. Invisible tant qu'un seul clip siégeait à la position 0, où les deux coïncident ; le surlignage désormais cloisonné le rendait manifeste. Backspace/Delete devenaient un no-op silencieux dès qu'un mot déjà coupé se trouvait sur leur trajet : la résolution tombait dessus et skipWordRange le jetait comme non conservé. La marche saute maintenant les mots déjà coupés, dans les deux sens ; une garde qui prétendait traiter le cas renvoyait le mot qu'elle venait d'écarter et était morte de toute façon. --- Suppression d'un trim = toute la pastille Une coupe étirée par-dessus une frontière de clip est nécessairement 2+ lignes (ventilateTimelineSpanToTrims) rendues comme une seule pastille. La suppression filtrait par id, laissant la moitié continuer de couper. `dropTrimPillsByIds` aligne les trois chemins (touche Suppr, inspecteur, outil LLM removeTrim) sur ce que dropPillById fait déjà pour les autres types ; le contournement local de FloatingInspector disparaît. Tests : aucun test existant ne couvrait deux clips partageant un asset avec des fenêtres source qui se chevauchent — le garde-fou le plus proche prenait des fenêtres à 45 s d'écart, donc il passait malgré le défaut. Chaque nouveau test a été vérifié en échec sans son correctif.
…ture path The feature list predated the AI chat and the transcript work, and the platform table still described Linux as browser-captured — which stopped being true when recording moved to the PipeWire helper. Linux now gets cursor themes and the editable overlay (the portal reports cursor shape); click effects stay Windows/macOS because the portal reports no button events. The webcam line was also wrong for macOS: only Windows muxes it natively, macOS records it alongside like Linux.
They were sized with style="height: 320px", which GitHub's sanitizer strips outright — verified against the /markdown API, which drops style and keeps width. The width attribute is what the logo above already uses. 48% each so they sit in one row. The two sources are 1.778 and 1.722, so equal width leaves a ~3% height difference; that reads far better than mismatched widths, and cropping someone's screenshots to fix it is not worth it.
Shows the current editor shell: AI chat panel, background picker, the reworked playhead and ruler.
1920x1080, matching demo.png's ratio to within a rounding error, so the two render at identical width AND height side by side. The previous take was 2:1 and left an ~11% height mismatch.
Le job « Typecheck (tests) » échouait : 75 erreurs pour un BASELINE de 66. Le dépassement est antérieur à cette PR — le tip vierge en comptait déjà 75 — quelqu'un a franchi le cliquet sans relever la baseline. Plutôt que relever, on corrige. Les 75 erreurs étaient presque toutes des fixtures auxquelles il manquait un champ requis par leur propre schéma : un asset sans `cameraTrack`, un clip sans `reason`, un document déclaré `AxcutDocument` mais de forme v3. Autrement dit des tests qui ne vérifiaient pas la forme qu'ils prétendaient vérifier — les compléter n'est pas cosmétique. Aucun fichier de production touché. Aucun `any`, `@ts-expect-error` ni `@ts-ignore` ajouté ; le diff est net-négatif de 15 casts (10 `as never` sur `tool.invoke`, 3 `as unknown as`, 1 `as AxcutDocument`). Aucune assertion supprimée ni assouplie ; deux tests se retrouvent renforcés — le fixture de `sceneDescription` reliait clip et asset par `undefined === undefined`, et `operations.test.ts` échoue désormais bruyamment sur un `sourceEndSec` absent au lieu de fausser un total en silence. Le cliquet disparaît, comme son propre commentaire le demandait (« delete this job's baseline logic at zero ») : le job devient un `tsc -p tsconfig.test.json --noEmit` sec. Il perd le tableau de résumé, et gagne un mode d'échec net. Les fixtures qui modélisent le contrat courant lisent maintenant la version sur `axcutSchemaVersion` au lieu de la redire. Une exception documentée sur place : dans `EditorEmptyState.test.tsx` le fixture est construit dans `vi.hoisted`, qui s'exécute avant l'initialisation des imports, donc la constante y est inaccessible et le littéral reste. Le titre de la PR est corrigé séparément : `rebase(...)` n'est pas un type Conventional Commits accepté par le job « Validate PR title ».
4afa3d4 to
2200c7c
Compare
Rebases the whole 1.8.0 line of work onto
main.386 commits · 884 files · +181,634 / −32,246 (611 added, 101 deleted, 158 modified)
LLM-driven editing chat
A chat panel that actually edits the document, not a decorative assistant.
electron/ai-edition/agent-tools.ts):addTrim/setTrim/removeTrim,setClipRange,moveClip,replaceTimeline,addZoom/setZoom,addSpeed/setSpeed,addAnnotation/setAnnotation,addCameraFullscreen/setCameraFullscreen,removeModifier,removeClip. The model gets a compact projection of the document (ids + times, three groups each with its own time base spelled out) rather than the raw document.provider-registry.ts): Claude API, OpenAI, Gemini, Mistral, OpenRouter, MiniMax (API and Token Plan), plus an OpenAI-compatible mode for an arbitrary endpoint. Two wire protocols (anthropic/openai), reasoning effort where the provider supports it, keys read from the environment or entered in the UI.createAgent(ReAct graph) — deliberately notcreateDeepAgent, whose empty filesystem sandbox led the model to fabricate proofs of absence.Local transcription (STT)
Everything runs on the machine; no network call to transcribe.
electron/native/whisper-stt/), built and packaged by the app — theggml-small-q8_0model is downloaded on demand from HuggingFace and cached.gpuDetector.ts): Metal on macOS arm64, Vulkan on Windows and Linux (CPU fallback at runtime), CUDA available, CPU binary everywhere else. ISA baseline pinned (Haswell/Zen 1 and newer) so the build doesn't ship a binary the host can't run.snapWordBoundaries), typed transcription contract.Multi-backend native compositor (78 commits)
SceneDescriptionas the app ↔ native contract, time-addressable rendering driven by the app's transport.Capture & recording
Duration.Editor & timeline
Export
UI & internationalization
design/): tokens, guidelines, reference components.Tooling
workbench/) for measuring the editing agent: scenarios, baselines, cassettes, CLI.THIRD-PARTY-NOTICES.md, CI workflow for the whisper build.Before merging
main:maincarries one extra commit,a7a52726 fix(ci): push the RC tag with the PAT, not GITHUB_TOKEN. Rebase onto it or let the merge reconcile it, depending on the strategy you pick.🤖 Generated with Claude Code